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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments filed 02 June 2008 have been fully considered but they are 
not persuasive. 

Applicant argues, "...movement detection as taught by Uchihashi would not 
benefit the system of Aoki because the system of Aoki detects conversations by 
analyzing audio data... the context of the teachings of Aoki detecting movement as 
taught by Uchihashi is irrelevant." (page 10, last paragraph). 

In response, Aoki provides motivation for adding Uchihashi's teaching in its 
invention. Aoki discloses that "one skilled in the art will understand that the invention 
can be augmented with additional known types of inferential analysis that use input data 
other than those directly affected by conversation... For example... context-awareness 
systems combine many kinds of physical sensor data and computer application data to 
make assessment of user activity. A context-aware system that is capable of tracking 
user's physical locations within a building can compute which users are co-present in a 
room; such a system might assess co-present users' vocalizations as having a high 
probability of being directed at each other as opposed to being directed at remote 
users... These other types of inferential analysis can be integrated with the present 
invention in a variety of ways. For example, they can be loosely integrated in a way that 
provides parameters that influence the operation of the 'floor configuration' thread... for 
manual user inputs... they can be tightly integrated, perhaps being incorporated directly 
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into a state machine that controls the operation of the present invention... The invention 
can be applied to any shared environment having independently controlled output." 
(pages 13-14, par. 0168-0169). Aoki also explains that its invention has a plurality of 
advantages, such as providing a unified conceptual and implementation framework for 
multi-party conversationally-responsive systems that can be applied to systems using 
different communication delivery media, such as audio, video and text (page 14, par. 
0178; read also: pages 5-6, par. 0071). 

Specification 

2. The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1 .75(d)(1 ) and MPEP § 608.01 (o). 

Correction of the following is required: Applicant claims a "computer-readable 
storage media" in claims 34-45, which is not defined in the specification. 

Drawings/Specification 

3. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they do not include the following reference sign(s) mentioned in the 
description: the description of figure 3 in the specification includes an element 13, 
which is not included in the drawing. Corrected drawing sheets in compliance with 37 
CFR 1 .121(d) are required in reply to the Office action to avoid abandonment of the 
application. Any amended replacement drawing sheet should include all of the figures 
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appearing on the immediate prior version of the sheet, even if only one figure is being 
amended. Each drawing sheet submitted after the filing date of an application must be 
labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 
CFR 1.121 (d). If the changes are not accepted by the examiner, the applicant will be 
notified and informed of any required corrective action in the next Office action. The 
objection to the drawings will not be held in abeyance. 

Applicant is required to amend the specification as needed to include any 
drawing corrections. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
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consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

6. Claims 1-8, 14-29, 32-41, 44 and 45 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Aoki et al (US 2004/0172255 A1) and Uchihashi et al (US 
2004/0201 71 0A1). 
Claims 1 and 22: 

Aoki discloses a system for communication (abstract; p. 1 , par. 001 1 ). Aoki 
discloses a first set of sensing and rendering components arranged to cover physical 
movements of multiple individuals present in a first environment. Aoki teaches a 
plurality of sensing components, such as microphones (p. 3, par. 0047) and vocalization 
detectors (p. 3, par. 0047); and rendering components, such as visual displays (p. 12, 
par. 0157; fig. 1), headphones (p. 14, par. 0170, lines 1-11), and audio speakers (p. 3, 
par. 0046), to cover physical activities and/or presence (tracking users' physical 
locations: p. 13, par. 0168, lines 8-10) of the individuals present in the environment. 
Aoki explains that the sensing and rendering components are used for detecting 
conversational characteristics (p. 4-5, par. 0061), such as user's physiological reactions 
(p. 9, par. 0114), user's vocalizations (p. 9, par. 0115), user's physical activity (p. 13, 
par. 0168), gestures such as mouse-clicks, button pushes, or voice commands directed 
at a computer (p. 14, par. 0174) of multiple individuals present in a (first environment) 
conversational floor (p. 1, par. 0012, 0013). Aoki teaches a context-awareness system 
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that may be combined with many kinds of physical sensor data and computer 
application data to make assessments of user activity; the system being capable of 
tracking user's physical locations within a building and compute which users are co- 
present in a room (motion or presence detection), (p. 13-14, par. 0168). 

Aoki discloses a second set of sensing and rendering components arranged to 
cover physical movements of multiple individuals in a second environment. Aoki 
teaches a system that enables groups and subgroups of people to communicate and 
carry on separate conversations within the context of a meeting (p. 1-2, par. 001 1 - 
0014). Aoki explains that the invention can include additional types of inferential 
analysis that use input data other than those directly affected by conversation. The 
system may combine many kinds of physical sensor data and computer application data 
(p. 14, par. 0170) to make assessment of user activity for users who are co-present in a 
room and/or users who are not present (first and second environments) in the room (p. 
13, par. 0168-0169). 

Aoki discloses interest thread detector that uses the first and second set of 
sensing and rendering components to detect multiple communication interactions each 
involving a respective subset of the individuals present in the first and second 
environments and that maintains an interest thread for each communication interaction. 
Aoki teaches detecting multiple communication interactions involving subgroups of 
individuals present in the same or in different environments (p. 13-14, par. 0168-0169). 
Aoki teaches maintaining an interest thread for each communication interaction (p. 14, 
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par. 0170). Aoki explains that the invention has a floor analysis module used to analyze 
data for conversational characteristics and that its results are used to determine users' 
activities, specify a set of mixing parameters, and communicate the mixing parameters 
over a floor control path (p. 4, par. 0056, 0061 ; p. 5, par. 0061-0062; p. 6, par. 0072, 
0073, 0076). 

Aoki discloses communication provider that captures a set of media data from 
the sensing components and that combines the captured media data in response to the 
respective activities of each subset of the individuals and that communicates the 
combined media data to the rendering components. Aoki teaches a communication 
system and a method for receiving a plurality of communications from a plurality of 
communication sources; for mixing the plurality of communication for a plurality of 
outputs associated with the plurality of communication sources (p. 1, par. 0011-0013; p. 
2, par. 0014-0015). Aoki teaches combining the captured media data and 
communicating the combined media data to the rendering components (p. 14, par. 
0173-0178). Aoki teaches a plurality of modules and a "conversational floor 
configuration thread" for analyzing conversational characteristics of the plurality of 
individuals (p. 4, par. 0056-0058, 0061, 0062), and using the results of the analysis to 
control the floor controls and/or set of mixing parameters. Aoki explains that the floor 
configuration thread is invoked by the "invoke floor configuration thread" and that this 
thread is responsible for receiving results of the individual analysis modules, 
determining and selecting the configuration of audio sources from these results (p. 6, 
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par. 0076). 

Aoki teaches analyzing data for conversational characteristics that can be 
determined from analysis of audio information or from physiological responses to the 
conversation, such as responses measured by a biometric device and/or information 
from an eye-tracker device (par. 01 14) but Aoki does not specifically teach detecting 
physical movement of the individuals. However, Uchihashi discloses a system and 
method for computer assisted recording or capture of meetings or presentation events 
(abstract; p. 1 , par. 0008). Uchihashi teaches a system having multiple cameras and 
sensors (p. 1 , par. 0017, 0019) for detecting physical motion of humans present in a 
conference (p. 2, par. 0022-0024, 0029; p. 4, par. 0046-0047). Uchihashi teaches 
displaying "candidate activity events", which are events of potential interest (interest 
thread) in the meeting (p. 5, par. 0051 ). Therefore, it would have been obvious to one 
ordinarily skilled in the art at the time the invention was made to modify Aoki's method 
of determining and analyzing the activities of multiple participants in a meeting to 
include Uchihashi's teaching of using sensors to capture physical movements of the 
participants because as Uchihashi says, the system follows the flow of the individuals' 
and/or the presentation's activities within the conference or meeting, and also because 
delays caused by camera switching latency is reduced or eliminated when the system 
captures only those activities that are of real interest to the meeting. 

Aoki/Uchihashi teaches combining captured media data in response to captured 
data in response to respective user's activities, and indicated by the user's shared 
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communication (page 14, par. 0169), conversational characteristics (page 14, par. 
0170), user's gestures (page 14, par. 0174), and other captured data such as user's 
physical activities (page 13, par. 0168). 
Claims 2, 23 and 35: 

See claim 1 . Aoki teaches selecting respective subsets of the first and second 
set of sensing and rendering components for use for each communication interaction 
(interest thread) or user's activities. Aoki explains that a meeting can have subgroups of 
people who carry on separate conversations, and that each subgroup can maintain 
awareness of the primary group conversation (p. 2, par. 0038). Aoki teaches a 
conversational floor wherein users are enabled to have side conversations in the 
conversational environment (p. 6-7, par. 0081-0082; p. 9-10, par. 0120). 
Claims 3, 24 and 36: 

Aoki teaches activities including speech levels of the individuals (user's 
vocalization) involved in the communication interaction (p. 3, par. 0035; p. 9, par. 0115, 
0118). 

Claims 4. 25 and 37: 

Uchihashi teaches activities including gestures by the individuals involved in the 
communication interaction (p. 3, par. 0035). 
Claims 5. 18. 20. 26 and 38: 

Uchihashi teaches activities including movements by the individual involved in a 
respective interest thread (p. 2, par. 0022-0024, 0029; p. 4, par. 0046-0047). 
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Claims 6, 27 and 39: 

Uchihashi teaches capturing and storing location information about identified 
objects and individuals within the conference room (p. 3, par. 0031 ; p. 4, par. 0043). 
Claims 7, 28 and 40: 

Aoki teaches refining captured data (p. 3, par. 0048). Uchihashi teaches 
processing the raw sensor information (p. 2, par. 0021; p. 4, par. 0045; p. 5, par. 0051). 
Claims 8, 29 and 41: 

Aoki/Uchihashi teaches storing captured data (Aoki: par. 0047; Uchihashi: par. 
0022, 0031, 0033). 
Claims 14-15. 32-33 and 44-45: 

See claim 1 . Aoki teaches capturing communication interactions (interest 
threads) of conversational activities of different groups of individuals (main conversation 
and side conversation) creating in this way interest area for separate detected activities 
(p. 6-7, par. 0081; p. 9-10, par. 0120, 0130; p. 13, par. 0168). 
Claims 16 and 17: 

See claim 1 . Aoki teaches communication interactions involving two or more 
individuals (p. 1 , par. 001 1 ) in two or more environments (p. 1 3-1 4, par. 01 68-01 69). 
Claim 19: 

Aoki/Uchihashi teaches movement pertaining to rendering devices (Uchihashi: p. 
2, par. 0020; p. 3, par. 0035). 
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Claim 21: 

See claim 1 . Aoki teaches participants may be in the same room or at a remote 
location (p. 13-14, par. 0168, 0170). 
Claim 34: 

See claim 1 . Aoki discloses a computer-readable media that containing code (p. 
4, par. 0050, lines 1-4). 

7. Claims 9-13, 30, 31, 42 and 43 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Aoki/Uchihashi and Rui et al (US 2002/01 91 071 A1 ). 
Claims 9-13. 30. 31.42 and 43: 

Aoki/Uchihashi teaches tracking interactions pertaining to an individual or an 
object (p. 2, par. 0020, 0031) but it does not teach communication interactions 
pertaining to an artifact that changes over time. However, Rui discloses a method for 
recording and broadcasting meetings (abstract; p. 1, par. 0009) wherein communication 
interactions that pertain to an artifact, such as a whiteboard, are captured and stored (p. 
1-2, par. 0010). Thus, it would have been obvious to a person having ordinary skill in 
the art at the time the invention was made to modify Aoki/Uchihashi's method of 
capturing communication interactions to include Rui's teaching of capturing 
communication interactions that pertain to devices because the participants are enabled 
to share a workspace, such as a whiteboard, and capture annotations that can be 
viewed by all participants in the conference room and then later reviewed by those 
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individuals who were or were not present in the meeting. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Valliath et al (US 7,106,358 B2) discloses a system and method 
for telepresence communication between two or more participants at multiple locations 
(abstract; col. 2, lines 58-67; col. 3, lines 1-15). Valliath discloses using multiple 
sensors for capturing, collecting and processing movement of participants (col. 3, lines 
16-67; col. 4, lines 1-37). Dietz (US 6,307,952 B1) discloses a system and method for 
detecting user's interactions. Dietz teaches using multiple motion sensors for detecting 
participants' movement (col. 3, lines 1-64; col. 4, lines 49-58). Waters et al (US 
6,256,046 B1) discloses a user interface that senses humans visually using movement 
and color to detect changes in the environment (abstract; col. 2, lines 25-39, 60-65). 
Waters teaches using multiple cameras for capturing people's movement (col. 3, lines 3- 
26) and speakers and display screens for rendering detected activity (col. 3, lines 27- 
63). 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
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MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to X. L. Bautista whose telephone number is (571) 272- 
4132. The examiner can normally be reached on Monday-Thursday 8:00AM-6:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Weilun Lo can be reached on (571) 272-4847. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

1 1 . Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
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USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/X. L. Bautista/ 

Primary Examiner, Art Unit 2179 



August 29, 2008 



